Skip to content

John6797/sc 7792/add raw data to validmind library tests#285

Merged
johnwalz97 merged 15 commits intomainfrom
john6797/sc-7792/add-raw-data-to-validmind-library-tests
Jan 14, 2025
Merged

John6797/sc 7792/add raw data to validmind library tests#285
johnwalz97 merged 15 commits intomainfrom
john6797/sc-7792/add-raw-data-to-validmind-library-tests

Conversation

@johnwalz97
Copy link
Contributor

@johnwalz97 johnwalz97 commented Jan 7, 2025

Internal Notes for Reviewers

Adds raw data to all tests. Also updates test unit tests to expect the new raw data in the tuple returned by the test.

External Release Notes

Adding raw data storage across all of our test library. Every ValidMind test now returns a RawData object which can be used in post-processing functions to recreate any test output. This feature continues the trend towards more flexibility and customizability.

@johnwalz97 johnwalz97 added enhancement New feature or request internal Not to be externalized in the release notes DO NOT MERGE PR is not ready to be merged and removed internal Not to be externalized in the release notes labels Jan 7, 2025
@johnwalz97 johnwalz97 removed the DO NOT MERGE PR is not ready to be merged label Jan 9, 2025
@johnwalz97 johnwalz97 force-pushed the john6797/sc-7792/add-raw-data-to-validmind-library-tests branch from 2ba31ac to 9d72f56 Compare January 9, 2025 20:04
Copy link
Contributor

@juanmleng juanmleng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool and extremely useful job @johnwalz97! Just left a minor comment. Otherwise looks good to go from my side.

Copy link
Contributor

@AnilSorathiya AnilSorathiya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks John. LGTM!

@github-actions
Copy link
Contributor

PR Summary

This pull request introduces significant changes to the AI test scripts and unit tests within the project. The primary focus of these changes is the integration of a new RawData mechanism, which allows for the storage and retrieval of intermediate data used in test functions. This enhancement is aimed at improving the post-processing capabilities and providing more detailed insights into the test results.

Key Changes:

  1. Removal of add_test_description.py:

    • The script add_test_description.py has been removed, indicating a shift in how test descriptions are managed or generated.
  2. Introduction of bulk_ai_test_updates.py:

    • A new script bulk_ai_test_updates.py has been added, which facilitates bulk updates on test files using AI. This script supports actions like adding descriptions and raw data to test files.
  3. Enhancements in Test Scripts:

    • Many test scripts have been updated to include the RawData class, which is used to store intermediate data that can be useful for regenerating test results during post-processing.
    • The RawData class is now imported and utilized across various test scripts, ensuring that raw data is captured and returned alongside traditional test outputs like figures and tables.
  4. Unit Test Updates:

    • Unit tests have been updated to accommodate the changes in test scripts, particularly the addition of RawData. This includes checking for the presence of RawData in test results and verifying its contents.
    • New unit tests have been added to ensure the correct functionality of the RawData integration.
  5. Refactoring and Code Quality Improvements:

    • The codebase has undergone refactoring to improve readability and maintainability. This includes reorganization of imports, removal of redundant code, and adherence to coding standards.
  6. Security and Dependency Management:

    • The changes include updates to ensure that all necessary dependencies are correctly imported and utilized, reducing the risk of runtime errors and improving the overall security posture of the codebase.

Test Suggestions

  • Verify that all test scripts correctly return RawData objects where applicable.
  • Ensure that unit tests for each modified test script check for the presence and correctness of RawData.
  • Run integration tests to confirm that the new bulk_ai_test_updates.py script functions as expected for different actions.
  • Test the removal of add_test_description.py to ensure no functionality is broken.
  • Check for any performance impacts due to the addition of RawData in test scripts.

@johnwalz97 johnwalz97 merged commit f180574 into main Jan 14, 2025
6 checks passed
@johnwalz97 johnwalz97 deleted the john6797/sc-7792/add-raw-data-to-validmind-library-tests branch January 14, 2025 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants